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ABSTRACT 



This report describes the Productivity Training Project 
(PTP) , a partnership between the University of Southern Colorado and Latino 
Chamber of Commerce (LCC) of Pueblo. The project provided "new economy 
skills" training to employees of LCC member businesses and basic skills 
training to the unemployed or underemployed. The report explains how PTP made 
progress in achieving most of its nine objectives: (1) it refined processes 

through which partnerships were established; (2) LCC's Corporate Advisory 
Council served as the Partners Council to oversee the project; (3) PTP 
customized programs of instruction to meet content and time recjuirements of a 
very wide variety of businesses; (4) PTP adopted a consultative approach to 
uncover workplace issues upon which to provide training; (5) it did not 
conduct job- task analyses; (6) it delivered new economy skills instruction to 
552 employees; (7) it served 762 individuals, 513 of whom achieved measurable 
gains in academic proficiency in one module of instruction or other training 
related goals; (8) it provided customized training for 45 LCC member 
businesses; and (9) PTP produced a final report. An evaluation of PTP based 
on interviews with 17 stakeholders categorizes seven themes that address 
project effectiveness into three groups: workplace education in the community 
(community and cultural context, business/education needs, barriers to 
providing workplace education) , program strengths (instructor expertise, 
impact of training) , and program challenges (marketing, past/future) . (YLB) 
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Tkis is tke final report to tke partners of tke Productivity Training Project, I kekeve we kave 
accompksked our overriding okjective as a demonstration project: to develop a puklic/private model for 
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component of a community's educational mix. 
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Productivity Training Project Final Report 



The Productivity Training Project (PTP) began in October of 1994 as a demonstration project 
of the U.S. Department of Education’s National Workplace Literacy Program (NWLP). The 
PTP has been a partnership between the University of Southern Colorado and the Latino 
Chamber of Commerce of Pueblo Colorado in the provision of "new economy skills" training to 
the employees of Latino Chamber member businesses and basic skills training to the unemployed 
or underemployed population of Pueblo. The following is a final report on the Project’s progress 
in achieving its nine major objectives (PTPOs). 



Productivity Training Project Objectives 

1) Design and implement an on-going process through the Latino Chamber of Commerce (LCC) 
for public/private partnerships for workplace literacy training in the city of Pueblo. 

Accomplished. From the outset of the partnership of the Latino Chamber of Commerce and the 
University of Southern Colorado in the PTP, the communication network among LCC member 
businesses has been essential to the ability of the Project to "sell" its services to the community, 
although we did not make effective use of this network until our second year of work. Initial 
questions about the relevancy of University delivered training to small businesses and questions 
about the quality of service being delivered by a new provider were significant barriers to securing 
commitments to participate from the LCC membership. Those businesses which supported the 
PTP idea through the proposal development stage were the first to take advantage of its services 
and sent the first positive messages through the LCC network. Through the first year, PTP 
marketing tactics were limited to standard print advertising and a universal message of the value 
of "new economy skills" training with a heavy reliance on the computer aided instructional 
method of imparting those skills. As the Project entered its second year, it became evident that 
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the packaging and advertising methods being used were hindering the progress of the Project 
toward meeting its qualitative and quantitative goals. We began using those business 
managers/owners who had already benefitted from the Project as our "sales force," and by relying 
on their referrals for initial contacts with potential partners, we increased the rate of successful 
enlistment of Project participants. At the same time, we abandoned the use of the word 
"curriculum" in our discussions of what the PTP had to offer businesses and focused our 
questions to potential partners on their business concerns. We discovered that for almost every 
business we contacted, we could add value by delivering training in the SCANS skills. Although 
we worked with businesses ranging from home healthcare providers to manufacturers of braking 
systems for large aircraft, the skills necessary to better address workplace productivity issues have 
been very much the same. In addition, the Latino Chamber has gone through a maturation of its 
ability as an organization to broker educational services that meet the needs bf its membership. 

The staff of the LCC has been expanded and encouraged by the Chamber Board to actively 
canvass the membership about what the workforce in Pueblo needs to improve its productivity. 
Also, the LCC performs the important function of clarifying with service providers, be they public 
such as the PTP or private contractors, cost, logistical, and follow-through requirements of 
service delivery to ensure that its membership receives the high quality it deserves. Over the 
three-year life of the PTP, we have refined the processes through which partnerships are 
established and the Latino Chamber of Commerce will maintain the practice of securing for its 
membership high quality educational services aimed at improving workplace productivity. The 
success of this process is further evidenced by the PTP’s accumulation of $326,000 in in-kind 
services, and salaried learner time over the three years of the Project. This greatly exceeds our 
obligated match target of $225,600. 
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2) Establish a Partners Council to oversee the project and curriculum committees at each 
company and/or training site. 

Accomplished. The nature and number of businesses we have worked with over the past three 
years have made it difficult to establish committees at each of the training sites. The majority of 
our training partners have been organizations of fewer than twenty-five employees, and the 
decision makers with whom we have worked to put programs into place often are 
owner/manager/line-workers whose time on-the-job is second in importance only to that of the 
full-time people on the production line. With some of the larger organizations for whom we have 
delivered training (the largest three employing roughly 900, 3(X) and 2(X) people), the PTP has 
been included in a mix of training initiatives established by the corporate management responsible 
for maintaining the skills of the workforce. In these cases we have had company committees 
assisting in the development of training programs. For the most part, we have reported to and 
received programming guidance from the Corporate Advisory Council of the Latino Chamber of 
Commerce. This group of representatives from twenty of Pueblo’s leading businesses meets 
quarterly and has served as our Partners Council over the past three years. They provide general 
oversight and guidance on Project management as well as important feedback on the effectiveness 
of the Project in meeting the needs of learners. The Partners Council has also been a valuable 
part of our marketing network through their advertising of the PTP in the numerous civic and 
professional organizations they represent. Their support has added tremendously to building the 
credibility of the PTP throughout the community, and their input has helped us ensure that our 
instructional offerings have been relevant to the needs of the community overall. With the help 
of the Partners Council, we defined and filled a niche in service delivery that we expect the Latino 
Chamber to continue to fill as it continues the Productivity Training Program after U.S.D.E. 
funding ends. In addition, a Joint Management Team (JMT) comprised of representatives of our 
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contracted service delivery partners, Pueblo Community College and Rocky Mountain SER, 
along with the president of the LCC and the director of the PTP was established and has been 
functioning as the operational oversight committee of the PTP throughout its three-year run. 

This group has worked on service delivery issues such as student retention, training site logistics, 
and equipment and materials efficiency. The open communication and sincere desire to see the 
PTP succeed on the part of each of these oversight groups has been an asset to the Project. 



3) Develop curriculum and acquire curriculum materials for a minimum of 20 job categories. 

Accomplished. The real strength of the PTP in meeting the needs of its training partners has been 
our ability to customize programs of instruction to meet the content and time requirements of a 
very wide variety of businesses. We have had the good fortune of employing professional training 
staff whose extensive business experience was constantly called into play as we delivered our classes 
or workshops. Our staffs thorough understanding of the nature of business environments was 
key to our ability to make relevant to working adult learners subjects in the SCANS 
Interpersonal competency area such as "Decision Making Styles" and "Conflict Management." 

We did not have to stretch much from theory to practice because we have first-hand knowledge of 
the value of new economy skills to the health of a business and the earning power of individuals. 
This contextual instructional delivery method was especially useful when our training partners’ 
business issues required instruction in the Resources and Systems competencies. For instance, by 
analyzing the daily activities of a small construction firm, our trainer was able to diagram the flow 
of materials against the time requirements for various concurrent jobs and lead the learners into 
discovering efficiencies that could allow them to do their work more productively and 
inexpensively. Also, this group of learners had not thought of themselves as a working system 
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which could better exploit its strengths for higher performance until their working processes were 
laid out for them in a more formal learning environment. Because of our work histories we know 
what to look for when gathering materials from the workplace to adjust otherwise standard 
academic material to reflect the realities of the organization. For instance, memos and letters 
written by officers, clerks, and the president of a local bank were used to modify an "Effective 
Workplace Writing" program developed by the Colorado Community College and Occupational 
System’s Educational Partnerships in Colorado for use in strengthening the writing skills of that 
bank’s employees. This is by no means a new approach to workplace education, but for the 
majority of the small businesses with whom we worked, the high degree of customization we 
provided was something they thought available only to large organizations whose size dictated that 
programs be built specifically for them. This preconception was sometimes a significant barrier to 
our recruiting participants in the program and was overcome primarily through the use of the 
LCC network to reinforce the Project’s credibility with potential partners. As we worked with 
more and more businesses, our repertoire of course titles became very large, but we consistently 
built our instruction around the SCANS competencies and foundations. 



4) Establish written goals for each company participating in the training, including collecting 
baseline data for evaluation of Project effectiveness. 

Accomplished. The small businesses with which we worked have been mostly entrepreneurial 
organizations which, because of the deep personal investment of the owners and managers, 
presented certain barriers to our performing traditional needs assessment types of activities in the 
process of establishing learning goals. Our small business partners have been generally reluctant 
to take part in formal assessments. Testing for skill levels was not an option the decision makers 
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cared to consider, nor were they very receptive to our observing their workplace to ascertain for 
what areas of their work an educational program would be appropriate. Again, trust in our ability 
to understand their business concerns was an issue we had to overcome before we could begin 
building with them a plan to address their educational needs. We found, however, that 
entrepreneurs will talk about their businesses--a lot. Our interviews with the owner/managers 
became the bases of our program development, and we adopted a consultative approach to 
uncovering those workplace issues upon which PTP provided training was likely to have an 
impact. We also discovered that these interviews were most effective when conducted in pairs, 
usually a PTP trainer and the Project director, because the interview process usually culminated 
with the business’ decision maker asking what we could do to help his or her organization deal 
with issues we had been discussing. This sort of on-the-spot program development required that 
either the trainer or the director draw out information about the business’ issues, workforce and 
past educational experiences while the other interviewer sketched out a plan of instruction to 
present in answer to the question, "What can you do for me?" This was our "point-of-sale." In 
most cases, whether or not the organization committed to and followed through with a PTP 
developed workplace educational program depended upon the answer we gave at that moment. 

The entrepreneurs running small businesses are used to making decisions quickly and investing 
themselves or their money in activities that they view as immediately relevant to their business 
objectives. If we did strike a cord during this initial interview, we were often asked to produce a 
more formal proposal for building the customized instruction for that business. Usually after one 
more meeting to discuss the formal proposal, the training would begin. Sometimes at the end of 
the initial meeting we were told where and when the workforce would be available, and we set dates 
and geared up to deliver six to twelve hours of instruction. The important learning that came out 
of this processes for the PTP stalf was that the businesses’ decision makers needed to be in 
control of why, how and when their workforce would take part in workplace education. We were 
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not successful in delivering a standard curriculum presentation for our market to choose from; 
however, we have been successful in allowing our market to identify their needs and draw from us 
what value we could impart to their employees. Understandably, we did not hit the mark every 
time with this process, but we were allowed to follow-up with instruction in areas beyond those 
outlined in our initial interviews in most situations where we discovered other needs of the 
organizations for which we provided training. When we gained the trust of the businesses we were 
in a better position to poll the workforce and mesh the sometimes differing opinions of the 
decision makers and workers into a more pointed, relevant and useful program of instruction. In 
all cases where our training has had an impact, the representatives of the businesses with whom we 
designed the program have been very much involved in ensuring the relevancy of the program to 
their business needs. 



5) Conduct job- task analyses (JTA) for a minimum of 10 job categories at 25 different 
companies. 

We did not meet this objective. The feedback we received from the LCC membership about why 
JTA was something the Project offered yet they did not choose to take advantage of was fairly 
consistent. Small businesses indicated that they could not see the value to their organization of 
taking the time to profile jobs. Although the SKILLS 2000 analysis tool that we had at our 
disposal greatly compresses the time necessary to perform a full job- task-analysis, we were unable 
to overcome the perception that the survey and debriefing process would cost the entrepreneurial 
business an inordinate amount of production time. On the other hand, the larger organizations 
to whom we advertised this service seemed to be heavily invested in the job profiling and ranking 
systems they had either created themselves or inherited from corporate human resource offices, 
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and they were unwilling to challenge what they had already done. Our efforts to perform formal 
assessments in a diagnostic fashion to use in building educational programs have been discussed 
above, and we did make an effort to tie JTA using SKILLS to this process; nonetheless, we were 
unable to establish a close enough relationship to any organization to build the profiles we set out 
in this objective. It may also be the case that our marketing of this service was somewhat lacking. 
We spent such a large part of our time in the personal selling of training that we may not have 
sufficiently explored the subtleties of making the connection between JTA and workplace 
efficiency. We did have a significant degree of success with the individual skill profiling 
component of the SKILLS package, however. We guided over sixty people through the process of 
building a personal skill inventory that they used as the basis for creating an educational plan for 
themselves, writing a skills-based resume, or choosing a career path more suited to their 
educational background. A number of people we have taken through this process have entered 
the university, community college, or new career fields. Though it may be tangential to our 
original objective, this outcome has been a productive one. The SKILLS 2000 analysis package 
will remain available to the community via the Latino Chamber of Commerce in its ongoing 
Productivity Training Program. 



6) Provide productivity-related training to a m ini mum of 600 employees from LCC member 
employers. 

The PTP will be forty-eight people short of accomplishing this objective. The Project has 
delivered new economy skills instruction to 552 employees of sixty-six LCC member employers 
This is inclusive of the customized training and seminars that were delivered in the first year of 
the Project when staff were still attempting to use the National Workplace Literacy Information 
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System (NWLIS) to track courses, learners, and contact hours. Most of the NWLIS data that 
was sent back to us in report format was unusable, so we have relied on our own hardcopy records 
to track participation from the beginning of year two. From October 1995 through March 
1997, we have served a total of 390 employees of thirty-six different Chamber member businesses 
in 127 separate instructional sessions. These sessions ranged from one to eight hours in length 
and took place during weekdays, weeknights, and weekends. As discussed above, the needs of 
these organizations defined much of how our instructional delivery took place. Through our 
open-enrollment skill improvement workshops for Latino Chamber member businesses, a total of 
ninety-one people from twenty-six different organizations attended the nine different series we 
have offered. These workshops were primarily lunch-time, brown-bag events, and over half of the 
ninety-one attendees participated in more than two of the workshops. Interestingly, there were 
some small businesses that were unwilling to take part in customized training for their 
organizations, but they sent employees to every one of our skill improvement workshops. In 
addition, these workshops served to support our recruitment efforts by giving businesses a sample 
of what we had to offer before they committed to working with us to build a program of their own. 
At least half of the organizations for which we built customized programs attended a skill 
improvement workshop before they invited us to work with their business. 



7) Provide workforce literacy training for a minimum of 650 area adults who are 
unemployed/underemployed as a result of skills shortfalls. 

Accomplished. The Project has served 762 individuals and 513 of those have achieved either 
measurable gains in academic proficiency in at least one module of instruction or other training 
related goals set out in their lEPs. This objective has been achieved primarily through the efforts 
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of our contracted training partners at Pueblo Community College and Rocky Mountain SER. 

Both the Community College and SER will be active participants in the One-Stop Career Center 
being developed for the Pueblo area. Officials from both organizations have participated actively 
in planning the One-Stop, and the successful collaboration we have accomplished through the 
FTP will serve as a model for processes that will be implemented in the One-Stop within the next 
year. Our pursuit of long-term value for the community in the area of adult basic skills has also 
led to the PTP having an impact on the shape of state-wide initiatives around adult basic 
education. The director of the Project was appointed to the State Board of Education’s Adult 
Literacy Commission and the Governor’s Business Commission on Adult Basic Learning where 
work is being done to rectily the State of Colorado’s lack of funding support for adult education. 
The Business Commission created a set of recommendations aimed at solidilying the State’s 
commitment to the preparedness of its workforce through legislated funding' of adult education 
programs and the continuation of the NWLP model of public/private partnership. The Adult 
Literacy Commission recently had a set of resolutions adopted by the State Board of Education 
that calls for much more active pursuit of funding sources, collaborative public/private 
partnerships, and integration of adult education services in Colorado Department of Education 
priorities. This work is leading toward a convergence of effort that will have significant positive 
effects on the adult education system in Colorado. 



8) Increase productivity in 40 LCC member businesses and confirm the relationship of Proj^t 
training to productivity gains using measures developed for each organization. 

Accomplished. The Project has provided customized training for forty-five different LCC member 
organizations. The challenging aspect of this objective has been developing measures of 
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productivity for each of the organizations we worked with. At various times and with various 
organizations we attempted to build metrics that would be useful in our assessment of training 
effectiveness. At the same time, we attempted to imbue the measurement tools we were creating 
with enough practicality to be of long-term use to the organizations for which they were being 
created. We met with very little success in developing quantitative measurement tools for our 
training partners. In investigating why this was so difficult, our external evaluation team cited 
"cultural" reasons. as the most likely hindrance to our success in this area. In writing about the 
broader barriers to delivering workplace education to our target population, the evaluators note 
that entrepreneurial culture seems to be "one where small business owners have made it on their 
own and are too proud to seek assistance or perhaps to share what they know about running their 
business. " This reluctance to share information may also have some rooting in the competitive 
nature of small businesses which requires small businesses to protect whatever edge they have as 
tightly held proprietary information. Our inability to measure our impact by-the-numbers 
notwit hstandin g, we were able to make definite conclusions about the value added by our Project’s 
efforts with the Latino Chamber membership. To confirm the relationship of PTP training to 
productivity gains, we relied upon the external evaluators’ qualitative analysis of our performance. 
They used an interview process to gather anecdotal information that was used to identify themes 
that speak to the Project’s effectiveness that may be applicable to other, similar workplace training 
efforts. (Please refer to the External Evaluation Report) 



9) Produce a comprehensive audit document and develop a final report suitable for 
dissemination, by September 1997. 



Accomplished. This final, expanded PTPO update and the External Evaluation Report are the 
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final pieces of the Productivity Training Project’s comprehensive, cumulative documentation of 
performance. We have produced PTPO updates on a bimonthly basis since January of 1995 
which have contained information about program changes, learner contacts, and ongoing 
assessments of training impact. In addition, the presentation of the PTP model that was 
included in the April/May 1997 PTPO update was disseminated at the national conference on 
workplace learning in April 1997. All records of Productivity Training Project activity will be 
warehoused at the University of Southern Colorado and the Pueblo Latino Chamber of 
Commerce staff will have access to any information related to the activities of the PTP over these 
past three years. 
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Productivity Training Project Report of Training Contacts 
Grant Year October 1996 - September 1997 
Totals As - Of; September 30, 1997 



PTP Office 

Businesses Served Participants 

38 202 



Individuals Served 
71 



Completions 

71 



Rocky Mountain SER 

Individuals Served Completions lEP Goals Met 

140 140 *125 



Pueblo Community College 

Individuals Served Completions lEP Goals Met 

119 119 *85 



* lEP section may not accurately reflect student goals due to missing data. 
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Productivity Training Project Report of Training Contacts 

Report Period 
08/01/97 - 09/30/97 



PTP Office 

Business Individuals Served Dates 

Mi Casa de Tranciciones 8 09/05, 09/26 



Individuals Served 
32 



Completions 

32 



Rocky Mountain SER 

Individuals Served Completions 

00 00 



lEP Goals Met 
*00 



Pueblo Community College 

Individuals Served Completions lEP Goals Met 

46 46 * 27 



* lEP section may not accurately reflect student goals due to missing data. 
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Productivity Training Project Report of Training Contacts 

Report Period 
06 / 01/97 - 07 / 31/97 



PTP Office 

Business 
BF Goodrich 
West Plains Energy 

Individuals Served 
20 



Individuals Served 
18 
15 

Completions 

20 



Rocky Mountain SER 

Individuals Served 
27 



Completions 

27 



Pueblo Community College 

Individuals Served 
36 



Completions 

36 



* lEP section may not accurately reflect student goals due to missing data. 
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Dates 

06 / 17 , 07/22 
06/24 



lEP Goals Met 
*12 



lEP Goals Met 
* 33 
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Productivity Training Project Report of Training Contacts 

Report Period 
04 / 01 / 97 - 05 / 31/97 



PTP Office 

Business 
BF Goodrich 
Bonfils Blood Center 
LCC Board of Directors 
Mi Casa De Transiciones 
Trane Company 



Individuals Served 
18 
8 

14 

8 

39 



Individuals Served 
10 



Completions 

10 



Rocky Mountain SER 

Individuals Served 
28 



Completions 

28 



Pueblo Community College 

Individuals Served 
12 



Completions 

37 



* lEP section may not accurately reflect student goals due to missing data. 




Dates 

04 / 22 , 05/19 

04/18 

04/18 

04 / 17 , 05 / 15 , 05/29 

04 / 29 , 05 / 01 , 05 / 06 , 
05 / 08 , 05 / 13 , 05/15 



lEP Goals Met 
*18 



TEP Goals Met 
* 25 
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Productivity Training Project Report of Training Contacts 

Report Period 
02/01/97-03/31/97 



PTP Office 


Business 


Individuals Served 


Dates 


BF Goodrich 


. 15 


02/13,03/20 


Bonfils Blood Center 


15 


02/14 


Central High School 


14 


02/07 


Cortez Construction 


04 


2/04 


Mi Casa De Transiciones 


07 


03/27 


Ruy-Gar 


17 


03/22 


SCA Insurance 


13 


02/12, 02/19, 03/07 


use Classified Staff 


07 


02/11 


use Accounting Department 


18 


03/26 


West Plains Energy 


15 


03/20 


Individuals Served 


Comnletions 




0 


0 




Rocky Mountain SER 


Individuals Served 


Comoletions 


lEP Goals Met 


56 


46 


*46 


Pueblo Community College 


Individuals Served 


Comoletions 


lEP Goals Met 


00 


0 


* 0 



* lEP section may not accurately refiect student goals due to missing data. 
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Productivity Training Project Report of Training Contacts 

Report Period 
12/01/96-01/31/97 



PTP Office 


Business 


Individuals Served 


Dates 


BF Goodrich 


15 


12/10,01/14 


Bonfils Blood Center 


16 


01/03,01/24 


Cortez Construction 


05 


12/30,01/23 


Kelly Services, Inc. 


03 


12/05, 12/12,01/16 


use Classified Staff 


09 


01/21 


Individuals Served 
3 


Completions 

*0 




Rocky Mountain SER 


Individuals Served 


Completions 


lEP Goals Met 


29 


22 


**22 


Pueblo Community College 


Individuals Served 


Comoletions 


lEP Goals Met 


25 


0 


** 0 



* Three individuals were served in open-enrollment workshops. Workshops will be completed in February. No lEP goals were established 
for participants in the workshops. 

** lEP section may not accurately reflect student goals due to missing data. 
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Productivity Training Project Report of Training Contacts 

Report Period 
10/01/96- 11/30/96 



PTP Office 


Business 


Individuals Served 


Dates 


BF Goodrich 


17 


11/05, 11/19 


Bonfils Blood Center 


12 


10/04, 11/01, 11/26 


Kelly Services, Inc. 


3 


11/07, 11/14, 11/21 


Latino Chamber Board 


11 


11/08 


Minnequa Bank 


6 


10/10, 10/15, 10/17, 10/22 


SCA Insurance 


10 


10/31,11/07,11/15 


use Classified Staff 


11 


10/31, 11/19 


Individuals Served 


Comnletions 




6 


♦6 




Rocky Mountain SER 


Individuals Served 


Comnletions 


lEP Goals Met 


29 


18 


♦*18 


Pueblo Community College 






Individuals Served 


Completions 


lEP Goals Met 


0 


0 


0 



* Six individuals were served in open-enrollment workshops. No lEP goals were established for participants in the workshops. 
•* lEP section may not accurately reflect student goals due to missing data. 
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Overview of External Evaluation Process and Methods 



The relationship between the Productivity Training Project (PTP) and the External Evaluation 
Team (consisting of Mary Crabbe Gershwin and Elizabeth Amidon Smith) has been an ongoing 
one characterized by collaboration. 

The evaluation team was contracted originally in 1996 to conduct the second-year evaluation of 
the project and re-contracted to conduct the final evaluation of the three-year project in February 
1997. Whereas one of the primary goals of the second-year evaluation was to help move the 
project into a successful third year under grant funding by consulting on strategic planning, the 
primary goals of the third-year evaluation are to assist the PTP in gauging its effectiveness as a 
workplace education program and to "provide guidance on what elements of the Project hold the 
most potential to be significant contributions to the learnings of NWLP demonstration grants" 
(PTP Objectives Update May 1997). These evaluative goals were defined in large part by the 
administration of the PTP. 

The evaluation team worked closely with the PTP to fulfill these objectives. Following what was 
mutually agreed upon as the success of the evaluative process of the previous year, the PTP and 
the evaluation team pursued a highly qualitative, collaborative and participative model for the 
evaluation. Qualitative methods employed by the team included interviews, document review 
and participation. The team made two site visits to Pueblo (in April and July 1997) for a total of 
three days to accomplish the following: 

=>Conduct 17 interviews with a representation of key stakeholders: 

PTP Project Director 

Latino Chamber of Commerce (LCC) Staff 

PTP Instructional Team 

University of Southern Colorado (USC) Dean 

Students 

Business Owners/Management 
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=^Coach and critique presentation "Delivering New Economy Skills Training to Small 

C 

Businesses" for the National Workplace Learning Conference in Milwaukeei WI 

=^Collect and gather information for the External Evaluation Report 

=^Review and reflect upon program progress which included presenting a draft of this 
External Evaluation Report to personnel of the FTP for feedback and input. 

Documents reviewed for this report included: PTP Objectives Update Reports to the USDE, 
Presentation Materials, letters and testimonials from businesses participating in the project, 
Proposal for collaboration and continuation of the PTP under the auspices of the USC Hasan 
School of Business. The evaluation team also attended the National Workplace Literacy 
Conference in Milwaukee, WI in April 1997 on behalf of the Productivity Training Project. 

What this report represents are stakeholders' perspectives of their experiences and of the 
effectiveness of the Productivity Training Project. In providing an analysis of these perspectives, 
the evaluators hope to present an understanding of this program by discussing themes that 
emerged from the data reviewed and by providing the,stakeholders with information that may be 
useful in moving forward with this educational program in their community. 
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Emergent Themes on the PTP's Effectiveness 

Interviews were conducted with 17 stakeholders. Several of these stakeholders held multiple 
roles, e.g. student and business owner or partner of the FTP and business owner. The interviews 
were open-ended with no standard (and the evaluators would argue limiting) interview protocol 
followed. The desire was to let themes emerge from the comments of the stakeholders on their 
perception of the project and their participation in it. Interviews started with an overall 
explanation of the purpose of the evaluation and an opening question such as "Please tell us 
about your involvement with the PTP" and the conversation would go from there. Time was the 
most obvious constraint on the process although the relationship with the evaluators was also a 
constraint on the comments and nature of the conversations entered into with each stakeholder. 

The analysis of the interviews with stakeholders as well as the document review revealed seven 
emergent themes which all speak to the effectiveness of this workplace learning project and 
perhaps to concepts about workplace projects that may be applicable to other programs: 

1) Community and Cultural Context 

2) Business/Education Needs 

3) Barriers to Providing Workplace Education 

4) Instructor Expertise 

5) Impact of Workplace Education 

6) Marketing of the PTP 

7) Past/Future of the PTP 

There is significant overlap in these seven thematic categories and we have chosen to group them 
as follows: 

Workplace Education in the Community: Community and Cultural Context, 
Business/Education Needs, Barriers to Providing Workplace Education 

Program Strengths: Instructor Expertise and Impact of Training 




5 



27 



Program Challenges: Marketing the FTP and Past/Future of the PTP 



Workplace Education in the Community 

At our follow-up visit and last evaluative visit for the second year of the grant, the group 
(evaluators, director, program assistant, instructors) discussed the nature of the community in 
which this project is situated. The question that everyone agreed would be worth looking at was 
"What is it in this community that would make a program successful?" 

The issue of community (and context of a program) was evident in the presentation that was 
developed for the National Workplace Conference as well as in many comments made in the 
interviews. Representatives of the Latino Chamber of Commerce (LCC) particularly stressed 
the characteristics of a community whose members want "to belong to something". Time was 
spent giving us information on the historical context of the economy in Pueblo. Those 
interviewed also wanted us to understand why education is so important in the community and 
what educational initiatives the LCC is promoting and supporting. 

It may have been valuable at the outset of this program to really work with the community to see 
what it wanted and needed in the realm of workplace education. One of the original partners 
said, "I thought when it started that everyone would beat down the doors", and this assumption 
that workplace education was something the community was waiting for was one that might have 
been examined at the outset. At the end of this project, the focus on community has led to a 
market study of the community's (and surrounding communities) business needs in order to 
assess workplace education's place and potential future. 

This question of the needs of the community and the needs of the businesses who participated 
was apparent in the interview comments — the majority of the interview comments were made on 
this topic and on the impact of the program. Business owners, managers and LCC representatives 
mentioned that companies don't know what their needs are and that a very important role the PTP 
played was to help them identify needs and then create educational programs that would address 
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those needs. Students needs were mentioned as well -- "the individual is the asset" (manager 
comment), "Where is leaming/human development?" (LCC staff questions), and one student 
commented, "I believe in the training and development of office staff. The tone of most of the 
conversations and the marketing materials of the FTP, however, stressed the needs of the 
businesses over the needs of individual students participating in these programs. Are the 
employees' needs subsumed by the business needs? The voice of student needs was not really 
heard. Overall the comments were very positive about the PTP’s abilities and sensitivity in 
recognizing and meeting the needs of the businesses; the presentation materials prepared on the 
PTP for the National Workplace Conference also presented this as one of the most significant 
skills in working with small businesses in particular. 

Barriers to providing workplace education were primarily three: time, money and culture. The 
cultural aspect was described as one where small business owners have made it on their own and 
are too proud to seek assistance or perhaps to share what they know about running their business. 
Time and money were cited by most of the business owners and managers interviewed as major 
constraints on their ability to provide educational opportunities for employees. This needs to be 
recognized for the barrier it is for future programs — the PTP was offering "free" services, the 
"cost" to businesses was release time for employees. 

Program Strengths 

The PTP in its three years had served 501 employees by June 1997; this speaks to the "success" 
of this project - moving from serving 2 business partners in their first year to 10 their second 
year to 61 by June of the third year. From the responses of interviewees and others involved with 
the PTP this is related to at least two major program strengths: the expertise and skill of 
instructional staff as well as the benefits/results/impact of workplace education felt in the 
businesses who took part. 
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As one person put it, "In this community, they buy the person". Instructor expertise was 
particularly critical to the success of the FTP and was valued and recognized by participants: 

Instructor was flexible to address issues. ..as they arose 

[The instructor's] real good as far as relating (with a diverse group) 

Enjoyed [the instructor] - great sense of humor 

I am convinced that [the instructor] is the finest TQM trainer — has the ability to bring it 
down to the level of staff.. .brought a lot of value to [the] FTP 

V/e attended several [brown bag lunches] and we were impressed with [the instructor] 
[The instructor's] been a professional resource for our company 

[The instructor] would always ask the audience how something might help, anyone have 
problems in this area? 

I've gone to a lot of seminars and [the instructor's] good at grabbing your attention 

[The instructor] adapted so well; [the instructor] always included himself 

As mentioned earlier, there were a large number of comments pertaining to the impact of 
educational programs. At the end of the second year of the project, the director was particularly 
concerned about measurement of program effectiveness (especially Level IV evaluation). Within 
the recommendations from last year's evaluation report, it was suggested that the FY? focus on a 
sample of the business partners they've worked with and evaluate success using measures that 
make sense to the business partner. It was also noted that anecdotal information from 
supervisors and employees to evaluate program success.. .is a valid source of evaluation 
information. One upper management participant who said he works in a measurable business 
said that, "We have truly improved.. .I'm not going to give FTP all the credit, but we’ve really 
improved". The means of measuring the success of a program or its organizational impact needs 
to make sense to the company or organization involved; the companies interviewed seemed to 
rely upon and prefer anecdotal feedback from staff on program success as opposed to numerical 
measures. 
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The responses of the interviewees would overwhelmingly indicate that this project in their 
perception had impact — whether on an individual level ("I saw that I could receive personal 
benefit”) or organizational level ("It facilitated change" "FTP prepared us to be successful and be 
prepared for change"). 

Program Challenges 

Was the FTP marketed sufficiently in the community? Did business know about the opportunity 
available to it? (These questions were raised in the interviews). It seems that the PTP with its 
difficult start (numerous personnel transitions) and its partnership with the Latino Chamber and 
use and the "turf battle" with Pueblo Community College had difficulty establishing a 
recognized and marketable identity. However, considering that the PTP surpassed its goal of 
providing workplace education to 40 members of the LCC by 50%, maybe the ability of this 
project to succeed despite the hurdles it faced speaks to its marketing ability. Participating 
businesses attested to this in the interviews - ""When they started the PTP, we were immediately 
contacted, and I was very impressed" "LCC sent a mailer, got my attention... got a follow-up call". 
The brown bag seminars were also credited as a strong marketing tool; they showcased the 
project to LCC members. 

. The PTP has worked hard to establish relationships and collaborations in order to transition as a 
grant-funded workplace education project to a community-based, self-supporting entity. The 
relationship with the LCC blossomed in the third year of the project. In part the housing of the 
project at the chamber's office building and increased exposure to businesses as well as the strong 
support and advocacy of the Board's chairman contributed to the strength of this relationship. 

The project director also worked particularly hard at establishing a link with the USC Hasan 
School of Business — a discussion paf>er was presented in summer 1996, a proposal presented in 
January 1997, and ongoing discussions and training offered to the department. The project 
should be commended on the efforts it has made to link with other entities in the community and 
to explore avenues of support, resource-sharing and future funding. It was clear in the discussion 
with the business school dean, however, that although verbal support of the project was strong, 
without the FTP being self-supporting financially, the school of business could offer little 
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material support other than office space. The LCC expressed interest and concern in "finding a 
way" to continue the project after the grant. 

The missing piece that is telling for workplace education projects is the commitment from the 
businesses where these programs operate. The interest and support from the' business community 
is what would sustain such a project. None of the owners or managers really commented on the 
future of the project, the partners in the PTP did. As one of the Latino Chamber representatives 
stated, "To me the Fl'P is a great benefit to the community, but we have to educate the 
community". 



Closing Remarks 

These remarks are written after the last evaluator visit to Pueblo where the draft of this report 
was presented to the director, program assistant and one of the instructors. The focus of the visit 
was to explain the methods used to develop this report, to present the analysis of the data and to 
reflect upon what meaning and understanding of the PTP has been constructed and how that fits 
with their experiences of this project. This visit was also an opportunity to discuss the plans for 
the future of the PTP. 

The director was comfortable with the qualitative focus of the report and said it was consistent 
with what the original proposal for evaluation had promised to provide. Our discussion for the 
most part initially focused around the "community piece", which the group agreed was "really 
important" to this project from its inception. It was mentioned that even in the context of writing 
the grant, the theme of "inclusion" was evident and was a unifying theme for this project 
throughout its years of activity. The hope was for this grant to help the community by being a 
community resource and to provide community members (under-employed, unemployed, 
chamber members and the University) with opportunities for involvement in education. As the 
director expressed, "My own personal agenda is to better everyone who participates". 

Much of the remaining discussion centered around what legacies were a part of the project and 
the hope that the project will remain a community resource. "Where will the computers go? 
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Where will they best suit the community?" Among the legacies of the FTP: the project has given 
the Latino Chamber of Commerce "credibility as a player" in educational programs (a major 
thrust of the LCC's mission for this year and coming years); the FTP has given the community a 
"benchmark" for quality educational instruction; and, hopefully, the FTP has on some level 
facilitated a change in individuals' and businesses' perspectives about education being an on- 
going pursuit. The director also saw his participation in the Governor's Commission on Adult 
Literacy as a significant contribution of the FTP in engaging the community in literacy issues on 
a statewide level. 

As the project is finishing up its activities under grant auspices, its director and staff are working 
to create a "bridge" for future activity. Two workshops are planned for the last two months of the 
project: a customer service seminar and a "leadership skills for women in business" series. The 
group is working to transition the Productivity Training Project to the Productivity Training 
Program under the Small Business Advisory Committee of the LCC. The LCC could continue 
to broker the services of the project to their business members and members would be expected 
to pay for the services provided. The instructors who have such credibility in the community 
could be hired to come in and deliver training. The concerns of the director are the marketing of 
the program which was a main part of his job and the quality and consistency of delivery which 
were hallmarks of the FTP. Linkages that would support membership of the LCC are being 
explored with the Small Business Development Co mmi ttee (SBDC) at Pueblo Community 
College as well as with the Small Business Institute at USC. 

Most significantly, the people who worked hard on this project in providing service to their 
community will continue to be advocates for adult workplace education in Pueblo. The expertise 
they brought to and gained from participation in the workplace grant will be felt in the 
community wherever they take their talents. As the director concluded, "Each of us who's been a 
productive piece of this is a member of the LCC and [is] active in the community in other 
contexts". 



Finally, the director requested that this report include reflection on how the FTP fits into the 
national movement of workplace learning. This evaluation team has worked with some twenty 
projects across the United States over the past three years, and given that national perspective, it 
appears that the FTP is unique in its success in reaching very small businesses (with less than 25 
employees). This small business sector has a great need for basic education for employees, but 
lacks both the resources to develop opportunities and training models that are appropriate for the 
demands in small business settings. Few workplace learning projects have successfully served 
small business. The PTP project has stepped into this gap by developing a practical model 
which weaves individual employee skill development together with small business management 
and planning. PTP’s model of combining training with the process of solving problems in small 
businesses presents a new solution to training the work force in small businesses. This solution 
demonstrates that training need not be an “add-on” or an extra that small business cannot afford 
in the rush of daily demands. Rather, it shows that learning can be a critical dimension to 
improving the process of operating a small business. This model was developed with federal 
funding for the cost of delivering training. It remains to be seen whether small businesses would 
fund this cost without external support. 
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